feat(tui): add interactive Terminal UI for mmore commands by perrin-arthur · Pull Request #298 · swiss-ai/mmore

perrin-arthur · 2026-05-08T15:51:49Z

Summary

Adds a mmore tui command that launches an interactive terminal interface built on questionary + rich. Users can:

Pick a pipeline command (process / postprocess / index / rag) from a menu
Reuse an existing YAML config or build one interactively (prompts for paths, models, options)
Run the full pipeline end-to-end with progress feedback and a banner UI

Lowers the barrier for non-CLI-savvy users who currently have to hand-write YAML configs.

Changes

New src/mmore/tui/ module:
- app.py — entry point, command picker, full-pipeline orchestration
- commands.py — registry of available commands
- config_builder.py — interactive config builder
- pipeline.py — runs a sequence of stages with rich-style progress
- theme.py — color tokens, banner, console helpers
src/mmore/cli.py — wires tui as a top-level Click command (mmore tui)
pyproject.toml — adds questionary>=2.0 and rich>=13 to tui deps
uv.lock — lockfile update for the new deps

Test plan

mmore tui launches the menu without errors
Selecting each command (process / postprocess / index / rag) walks through the config builder
Reusing an existing YAML config works
Running the full pipeline reports per-stage progress and final status
pytest still green (no test changes in this PR)

Execution

If not already done, install the dependencies:

uv pip install -e

Then launch the CLI:

mmore tui

Adds a `mmore tui` command that launches an interactive terminal interface built with `questionary` and `rich`. Lets users pick a pipeline command (process / postprocess / index / rag), reuse or build a config interactively, and run the full pipeline with progress feedback — without having to write YAML configs by hand. - New `src/mmore/tui/` module (app, commands registry, config builder, pipeline runner, theme). - Wires up `tui` as a top-level Click command in `cli.py`. - Adds `questionary>=2.0` and `rich>=13` to core dependencies.

Copilot

Pull request overview

Adds an interactive mmore tui command (questionary + rich) to help users pick and run mmore pipeline stages with guided config selection/generation and richer terminal feedback.

Changes:

Introduces a new src/mmore/tui/ package implementing menus, config prompting/validation, and a “full pipeline” runner.
Wires a new top-level Click command mmore tui into src/mmore/cli.py.
Adds questionary and rich as core dependencies (with corresponding lockfile updates).

Reviewed changes

Copilot reviewed 8 out of 9 changed files in this pull request and generated 5 comments.

Show a summary per file

File	Description
uv.lock	Locks new runtime deps needed for the TUI (`questionary`, `rich`, transitive deps).
pyproject.toml	Adds `questionary>=2.0` and `rich>=13` to core dependencies.
src/mmore/cli.py	Adds the `tui` Click command entrypoint.
src/mmore/tui/init.py	Exposes `run()` for the CLI entrypoint.
src/mmore/tui/app.py	Main TUI loop: menu selection and command execution UX.
src/mmore/tui/commands.py	Registry mapping TUI options to existing `run_*` functions and config validation dataclasses.
src/mmore/tui/config_builder.py	Guided config generation + config picking + YAML validation loop.
src/mmore/tui/pipeline.py	Orchestrates process → postprocess → index (+ optional chat) with progress UI and summary.
src/mmore/tui/theme.py	Shared Rich styling helpers (banner/panels/colors).

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

- Centralise QSTYLE/QMARK in theme.py (was duplicated in app.py and config_builder.py) - Derive pipeline output paths via load_config + jsonl_path so env-var expansion ($ROOT_OUT_DIR, ...) and the directory-vs-jsonl logic match what the underlying commands actually use - Add paths.py: repo_root() walks up from CWD to find examples/, so the TUI works from any working directory; cwd_default() gives ./data-style defaults instead of repo-relative paths - Replace examples/... defaults in guided prompts with cwd_default() fallbacks so the TUI is sensible from outside the repo - ruff format pass on the tui/ package

Gsharpp

1. The TUI doesn't detect missing extras

Repro: uv sync (no extras) → mmore tui → pick process → run → ModuleNotFoundError: No module named 'torch'. The TUI opens happily because questionary and rich are in core deps, but the pipelines it offers depend on the process / index / rag / cpu|cu126 extras, which aren't guaranteed to be installed.

We need to check that the required modules are importable before launching a stage, and surface the matching uv sync --extra <stage> --extra cpu command. Ideally: gray out (in the menu) stages whose extra isn't installed.

2. `questionary` + `rich` declared as `dependencies` instead of an extra

This joins the previous paragraph. We may prefer the TUI functionality to be optional in the dependencies

3. `KeyboardInterrupt` used to signal "sub-prompt cancellation"

Pressing ctrl + C exit the full tui application. There is no current signal to cancel a command while staying in the app.

4. Better YAML creator function

There is currently no wizard to create yml config files. This might be great to create several templates of configuration files that the wizard could use to help the user

5. No documentation about this integration

No mention of the TUI functionality in the README

- Add a guided wizard ("Build a full pipeline config") that generates coherent process + postprocess + index YAMLs in one flow, exposing only processors / post-processors / indexer types that actually exist in the repo (pulled from ProcessorRegistry, TAGGER_TYPES, FILTER_TYPES). - Detect missing extras per stage via importlib.util.find_spec canaries; disable menu entries and surface the exact `uv sync --extra ...` hint instead of crashing mid-run with ModuleNotFoundError. - Move questionary + rich out of core dependencies into a new `tui` extra (included in `all`); friendly error from `mmore tui` if missing. - Introduce CancelledByUser so Ctrl-C / Esc inside a sub-flow returns to the main menu instead of exiting the whole TUI. Ctrl-C at the main menu still quits. - Add a spinner during YAML validation (dataclass imports take ~5s and made the TUI look frozen). - Document the TUI in the README with install commands and behavior.

…o feat/tui

…OR edit - Add post-validation menu (preview / edit / run) when picking a config: syntax-highlighted YAML preview via rich.Syntax, and $EDITOR launch with automatic re-validation on save. - New inspector module (src/mmore/tui/inspector.py) that streams JSONL output files and prints a summary table (doc count, processor types, file types, avg text length, modalities) plus a sample of the first 3 documents. Called automatically after process and postprocess steps. - Wizard builders now detect existing output files and propose resuming via previous_results instead of always writing null, leveraging the existing incremental.py module for skipping unchanged files. - Add "Edit an existing YAML in $EDITOR" choice to pick_or_build_config.

Copilot

Pull request overview

Copilot reviewed 13 out of 14 changed files in this pull request and generated 9 comments.

+"""Locate bundled example configs regardless of CWD or install layout.
+
+Strategy:
+- If `examples/` exists relative to CWD (source checkout), use it.
+- Else, walk up from CWD looking for a repo root that contains `examples/`.
+- Else, fall back to `importlib.resources` to read examples shipped with the
+  package (only available if the wheel actually bundles them).
+- If nothing is found, return the original repo-relative path so error
+  messages stay readable; callers handle "missing" gracefully.
+"""


+    """Open a config file in $EDITOR (falls back to vi)."""
+    editor = os.environ.get("EDITOR", "vi")
+    subprocess.call([editor, path])
+


+    try:
+        from .tui import run
+    except ImportError as e:
+        click.echo(
+            f"TUI dependencies missing ({e.name or e}). "
+            "Install with: uv sync --extra tui"
+        )
+        raise SystemExit(1)


- Use time.time_ns() for config filenames to avoid collisions - Support $EDITOR with flags (e.g. "code -w") via shlex.split - Use _validate_with_spinner in pick_or_build_config to show feedback during slow dataclass imports - Expand ~ and env vars on manual path input (expanduser/expandvars) - Replace cwd_default("examples/...") with resolve_example() so defaults resolve correctly from any CWD - Narrow ImportError catch in cli.py to ModuleNotFoundError for expected TUI deps only, re-raise other import errors - Fix paths.py docstring to match actual implementation (no importlib.resources fallback)

Add a guided setup flow accessible from the main menu that walks the user through picking pipeline stages, selecting a compute backend (cpu / cu126), running `uv sync` with the right extras, and generating a .env file with the API keys / paths each stage needs. Existing .env entries are preserved on merge, and secret values are masked in the preview table.

Outline characters in white, filled blocks in pure black, second M in yellow — matches the m(m)ore logo. Uses hex colors to avoid terminal themes remapping ANSI black to dark grey.

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Copilot <copilot@github.com>

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: fabnemEPFL <117652591+fabnemEPFL@users.noreply.github.com>

JCHAVEROT

Hi @perrin-arthur & @Gsharpp,

I tested the interactive CLI and it looks really good 👏 I was able to run all the commands, and it’s great to have such a user-friendly interface. The setup took me a bit of time at first, but I added a comment about that.

I only left a few comments, and the fixes should be quick. I mainly reviewed the documentation and src/mmore/tui/app.py, which defines the functions being called. For the rest of the tui/ folder, since it is fairly independent from mmore and there is no risk of introducing bugs into the pipelines, I’ll trust you on that part as I couldn’t review it in depth (it’s very TUI-specific and the code looks clean overall).

So you know I had to rebase and merge origin/master in your branch because otherwise I couldn't run the index and RAG pipelines (there was a dependency fix needed), hence don't forget to run a git pull before your changes

- wire run_setup_wizard() in app.py dispatch (elif mode == "setup") - add uv to tui extra in pyproject.toml so setup wizard can run uv sync - replace .env file generation with export command hints (mmore does not use dotenv for secrets; exporting is simpler and safer) - remove ({e.name}) from cli.py TUI missing-dep error message - simplify README install snippet to uv sync --extra tui; remove implementation detail paragraph (already covered in for_devs.md)

JCHAVEROT

Thanks for the changes 👍

fabnemEPFL

Just a little thing to fix but sounds great overall!

fabnemEPFL · 2026-05-29T09:52:53Z

+    config_globs: list[str] = field(default_factory=list)
+    # Lazy importer returning the dataclass to validate YAML against.
+    # Returns None if no validation is wired up for this stage.
+    config_dataclass: Optional[Callable[[], Any]] = None


very weird type hint, should be something like Optional[Type[BaseConfig]] instead

fabnemEPFL

I'm approving this PR now and you will fix typing with a new PR on another day

Copilot AI review requested due to automatic review settings May 8, 2026 15:51

Copilot started reviewing on behalf of perrin-arthur May 8, 2026 15:52 View session

Copilot AI reviewed May 8, 2026

View reviewed changes

Comment thread src/mmore/tui/pipeline.py

Comment thread src/mmore/tui/pipeline.py Outdated

Comment thread src/mmore/tui/commands.py

Comment thread src/mmore/tui/config_builder.py

Comment thread src/mmore/tui/app.py Outdated

Gsharpp reviewed May 11, 2026

View reviewed changes

perrin-arthur and others added 7 commits May 12, 2026 08:50

fix(tui): rename CancelledByUser→UserCancelledError, apply ruff format

628ca18

Merge branch 'swiss-ai:master' into feat/tui

b2bae0a

fix: adding wizard config for single command

9d996fa

Merge branch 'feat/tui' of https://github.com/perrin-arthur/mmore int…

09589e6

…o feat/tui

docs: for_devs updated

c303eee

fabnemEPFL requested a review from Copilot May 12, 2026 13:51

Copilot started reviewing on behalf of fabnemEPFL May 12, 2026 13:51 View session

Copilot AI reviewed May 12, 2026

View reviewed changes

Gsharpp and others added 14 commits May 12, 2026 16:38

disable commands when pipeline extras are missing | big fix

f521702

fix(tui): expose merged_results_path helper in run_process + uv.lock

bb80849

remove spinner + expose merged_results_path helper

244a972

warm pipeline dataclasses + factor run_step helper + update .gitignore

072901b

style(tui): match GitHub logo colors in banner

47b8b12

Outline characters in white, filled blocks in pure black, second M in yellow — matches the m(m)ore logo. Uses hex colors to avoid terminal themes remapping ANSI black to dark grey.

fix(tui): ruff lint and format

ca5479d

center banner, centralize _ask, add int/float prompts

a7fe73f

Update paper link from OpenReview to arXiv

c7f67c4

Fix tests not passing in CI (swiss-ai#304)

d3fd965

Fix consumed file ID when upload fails (swiss-ai#299)

c5415a2

Fix swiss-ai#288 (swiss-ai#307)

ca412b1

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: Copilot <copilot@github.com>

Add workflow for Pyright type check (swiss-ai#300)

4860624

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Co-authored-by: fabnemEPFL <117652591+fabnemEPFL@users.noreply.github.com>

Merge remote-tracking branch 'origin/master' into feat/tui

3bbf489

JCHAVEROT reviewed May 28, 2026

View reviewed changes

Comment thread src/mmore/tui/setup.py Outdated

Comment thread src/mmore/tui/setup.py Outdated

Comment thread src/mmore/cli.py

Comment thread README.md Outdated

Comment thread README.md Outdated

JCHAVEROT added documentation Improvements or additions to documentation enhancement New feature or request dependencies Pull requests that update a dependency file labels May 28, 2026

JCHAVEROT assigned perrin-arthur and unassigned perrin-arthur May 28, 2026

JCHAVEROT approved these changes May 28, 2026

View reviewed changes

Comment thread src/mmore/tui/app.py Outdated

perrin-arthur and others added 2 commits May 28, 2026 18:51

Update setup choice text in app.py

1a68bd8

chores: fix linter

25fcd5a

fabnemEPFL requested changes May 29, 2026

View reviewed changes

fabnemEPFL approved these changes May 29, 2026

View reviewed changes

fabnemEPFL merged commit 94192f1 into swiss-ai:master May 29, 2026
4 checks passed

Conversation

perrin-arthur commented May 8, 2026 • edited by fabnemEPFL Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Test plan

Execution

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Gsharpp left a comment

Choose a reason for hiding this comment

1. The TUI doesn't detect missing extras

2. questionary + rich declared as dependencies instead of an extra

3. KeyboardInterrupt used to signal "sub-prompt cancellation"

4. Better YAML creator function

5. No documentation about this integration

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JCHAVEROT left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

JCHAVEROT left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

fabnemEPFL left a comment

Choose a reason for hiding this comment

Uh oh!

fabnemEPFL May 29, 2026

Choose a reason for hiding this comment

Uh oh!

fabnemEPFL left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

perrin-arthur commented May 8, 2026 •

edited by fabnemEPFL

Loading

2. `questionary` + `rich` declared as `dependencies` instead of an extra

3. `KeyboardInterrupt` used to signal "sub-prompt cancellation"

JCHAVEROT left a comment •

edited

Loading

fabnemEPFL left a comment •

edited

Loading